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Closing Achievement Gaps With a Utility-Value Intervention: 
Disentangling Race and Social Class 

Judith M. Harackiewicz, Elizabeth A. Canning, Yoi Tibbetts, Stacy J. Priniski, and Janet S. Hyde 

University of Wisconsin-Madison 


Many college students abandon their goal of completing a degree in science, technology, engineering, or 
math (STEM) when confronted with challenging introductory-level science courses. In the U.S., this 
trend is more pronounced for underrepresented minority (URM) and first-generation (FG) students, and 
contributes to persisting racial and social-class achievement gaps in higher education. Previous inter- 
vention studies have focused exclusively on race or social class, but have not examined how the 2 may 
be confounded and interact. This research therefore investigates the independent and interactive effects 
of race and social class as moderators of an intervention designed to promote performance, measured by 
grade in the course. In a double-blind randomized experiment conducted over 4 semesters of an 
introductory biology course ( N = 1,040), we tested the effectiveness of a utility-value intervention in 
which students wrote about the personal relevance of course material. The utility- value intervention was 
successful in reducing the achievement gap for FG-URM students by 61%: the performance gap for 
FG-URM students, relative to continuing generation (CG)-Majority students, was large in the control 
condition, .84 grade points ( d = .98), and the treatment effect for FG-URM students was .51 grade points 
(d = 0.55). The UV intervention helped students from all groups find utility value in the course content, 
and mediation analyses showed that the process of writing about utility value was particularly powerful 
for FG-URM students. Results highlight the importance of intersectionality in examining the independent 
and interactive effects of race and social class when evaluating interventions to close achievement gaps 
and the mechanisms through which they may operate. 
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Many students start college intending to pursue a career in 
science, technology, engineering, or math (STEM), but too many 
abandon this goal after introductory courses, either because they 
perform poorly, lose interest, feel uncomfortable in the course, or 
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some combination thereof. Some groups are at greater risk for 
these problems. For example, African Americans, Hispanics, and 
Native Americans together constitute 26% of the U.S. population, 
but only 9% of STEM professionals, and 11% of STEM degree 
recipients in 2008 (National Science Boat'd, 2012). Another group 
that struggles in college is first-generation (FG) students, those for 
whom neither parent obtained a 4-year college degree, compared 
with continuing-generation (CG) students, who have at least one 
parent with a 4-year degree. FG students constitute roughly 20% of 
students in American universities and represent a potentially large 
STEM talent pool, yet they drop out of college at a higher rate 
(28%-35%) than CG students (17%; Chen, 2005; Radford, 
Berkner, Wheeless, & Shepherd, 2010). If we wish to increase the 
number of students in science and maximize the chances of dis- 
covering talent, it is critically important to promote motivation and 
performance for underrepresented ethnic minority (URM) and FG 
students in introductory science courses, which act as a gateway to 
STEM careers (Ferrini-Mundy, 2013). 

Differences in academic performance between URM and ma- 
jority students are referred to as racial achievement gaps, whereas 
differences between FG and CG students are referred to as the 
social-class achievement gap, because parental education is a 
proxy for socioeconomic status (Fiske & Markus, 2012; Jackman 
& Jackman, 1983; Pascarella & Terenzini, 1991; Snibbe & 
Markus, 2005). These achievement gaps can be attributed to a 
number of economic and social factors such as poverty, quality of 
schools, economic resources, and academic preparation (Bowen, 
Kurzweil, & Tobin, 2005; Pascarella & Terenzini, 1991), but they 
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may also reflect psychological factors to the extent that FG and 
URM students feel less engaged in their classes, feel stigmatized, 
or worry about whether they belong in the course or at the 
university (Johnson, Richeson, & Finkel, 2011; Ostrove & Long, 
2007). 

Brief Interventions 

In recent years a number of social-psychological interventions 
have been developed to help at-risk students in introductory col- 
lege classes, with striking effects (Yeager & Walton, 2011). These 
brief interventions are powerful because they are psychologically 
precise and focus on a specific problem (e.g., identity threat, 
disengagement) at critical time points (e.g., freshman orientation, 
gateway courses; Walton, 2014). Some interventions, such as the 
values affirmation and belonging interventions (Cohen, Garcia, 
Apfel, & Master, 2006; Walton & Cohen, 2011) are domain- 
general, and focus on students’ self-beliefs and sense of belonging 
in college. Others, such as the utility-value intervention (Hulleman 
& Harackiewicz, 2009), are domain-specific and address factors 
such as task values or interest in a particular course. These inter- 
ventions have been used to help different groups of students. Some 
interventions focus on students with a history of poor performance 
(Hulleman, Codes, Hendricks, & Harackiewicz, 2010), whereas 
others have targeted either racial gaps (e.g., Cohen et al., 2006; 
Sherman et al., 2013; Walton & Cohen, 2011) or social class gaps 
in academic achievement (e.g., Harackiewicz, Canning et al., 
2014; Stephens, Hamedani, & Destin, 2014). 

In this research, we tested two brief writing interventions hy- 
pothesized to help underrepresented students in an introductory 
college biology course: the values affirmation (VA) and utility- 
value (UV) interventions, which work at different levels of domain 
specificity and address different problems. The VA intervention 
targets one obstacle to student achievement, namely, identity 
threat. Students can feel threatened when they are aware of ste- 
reotypes about their group or worry about whether they “fit in” at 
college, and this can undermine their performance (Cohen, Purdie- 
Vaughns, & Garcia, 2012; Ostrove & Long, 2007; Stephens, 
Fryberg, Markus, Johnson, & Covarrubias, 2012). When individ- 
uals affirm their personal values in a threatening environment, 
however, they can reestablish a perception of personal integrity 
and worth (Cohen & Sherman, 2014). For example, Cohen, Garcia, 
Apfel, and Master (2006) found that a brief writing exercise in 
which students reflected on their core personal values reduced the 
gap in grades between African American and European American 
students by almost 40%. More recently, Harackiewicz, Canning et 
al. (2014) found that the same VA technique worked to close the 
social-class achievement gap in college biology. In contrast, the 
UV intervention has not previously been tested with respect to 
racial or social class gaps, but has been shown to help students 
with a history of poor performance achieve higher grades in their 
classes (Harackiewicz & Hulleman, 2010). 

Utility-Value Interventions 

The UV intervention targets different psychological processes 
critical to student achievement: perceived value of and engage- 
ment in coursework. It is a curricular intervention in which stu- 
dents write short essays about the personal relevance of course 


material. For example, a student might write an essay about how 
what she learned about animal physiology informs her future 
workout plans, explaining the basic principles of muscle potential 
and relating them to her life. These course assignments help 
students discover connections between course topics and their 
lives. The intervention is based in expectancy-value theory (Eccles 
& Wigfield, 2002), which argues that individuals choose to take on 
challenging tasks — such as declaring a STEM major or persisting 
in a difficult biology course — if they (a) value the task, and (b) 
expect that they can succeed at the task (based on self-beliefs). 
Beliefs about the self (e.g., “I am very good at science”) and 
beliefs about the value of the task (e.g., “Biology is an important 
field”) are both critically important in predicting academic moti- 
vation. However, it may be more feasible to influence students’ 
subjective task values than their expectations for success (Pajares, 
1996). Eccles (2009; Eccles et al., 1983) identified four types of 
subjective task values: intrinsic value , the perceived importance of 
a task because of its inherent enjoyment or interest; attainment 
value , the perceived importance of a task for an individual’s 
identity and self-worth; utility value, the perceived importance or 
usefulness of a task for accomplishing future goals; and cost value, 
the perceived negative aspects of engaging in a task (e.g., time 
consumption). Expectancy-value theory posits that an increase in 
any of these values (except cost) will lead to greater motivation 
toward an academic task. 

Of these three positive task values, Eccles and colleagues con- 
sider utility value to be the most “extrinsic” because it extends 
beyond the task itself to connections between that task and other 
tasks, activities, or goals (Wigfield & Eccles, 1992), and it may 
therefore be the task value most responsive to external interven- 
tion. In educational contexts, a student finds utility value in a topic 
if they believe it is useful and relevant beyond the immediate 
situation, for other goals or aspects of their life. Correlational 
research documents that when students perceive utility value in 
their courses, they work harder, develop more interest, and per- 
form better (Brophy, 1999; Harackiewicz, Durik, Barron, 
Linnebrink-Garcia, & Tauer, 2008; Hulleman, Durik, Schweigert, 
& Harackiewicz, 2008; Wigfield, 1994). Moreover, recent exper- 
imental research indicates that it is possible to promote perceived 
utility value with simple interventions that ask students to write 
about the relevance of course topics to their own life or to the life 
of a family member or close friend (Canning & Harackiewicz, 
2015; Hulleman et al., 2010; Yeager et al., 2014). This leads 
students to discover connections between course topics and their 
lives, in their own terms. 

Discovering these connections helps students appreciate the 
value of course work, and can promote a deeper level of cognitive 
engagement (Harackiewicz & Hulleman, 2010). In other words, 
the intervention works by changing how students think about 
course topics. In addition, we know from experimental laboratory 
work (Canning & Harackiewicz, 2015) that self-generated utility- 
value information (i.e., utility-value connections that students 
identify for themselves) is more powerful than externally commu- 
nicated utility-value information (as might be produced, e.g., when 
a professor tells students that material is important and relevant). 
The key is having students work to find the utility value for 
themselves, which is facilitated through writing assignments that 
are central to the intervention. 
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The utility-value intervention can promote academic perfor- 
mance by fostering student engagement with course content, and 
by helping students find personal value in the material. This 
intervention works best for students who doubt their competence 
and for those with a history of poor performance. Students who 
struggle in classes or doubt their competence are at greater risk for 
disengagement with course content (Durik, Hulleman, & Harack- 
iewicz, 2015). However, Hulleman and Harackiewicz (2009) 
found that their UV intervention raised interest and grades for 
ninth grade science students who had low performance expecta- 
tions in the course, relative to students in a control group who 
wrote summaries of course topics. Hulleman, Godes, Hendricks, 
and Harackiewicz (2010) found that a UV intervention promoted 
interest in an introductory psychology class for students who had 
performed poorly on early exams, relative to a control group. They 
also showed that the UV intervention promoted students’ percep- 
tions of utility value in the psychology course, and that this 
increase in perceived utility value then increased interest, intention 
to major, and grades. 

Can Utility- Value Interventions Close 
Achievement Gaps? 

Although the UV intervention has proven effective in increasing 
motivation and academic performance for high school and college 
students who had low success expectancies and/or low perfor- 
mance early in the course, it has not previously been imple- 
mented to close racial or social-class achievement gaps. We 
hypothesize that both URM and FG students will benefit from 
a UV intervention relative to majority and CG students, in much 
the same way that students with a history of low performance 
benefit from these interventions. The UV intervention might prove 
effective for all students with a history of poor performance, 
whether URM, FG or not, in which case its efficacy would be 
attributed to its power to engage students who struggle academi- 
cally (Durik, Hulleman, & Harackiewicz, 2015), regardless of 
ethnicity or generational status. Alternatively, the intervention 
might have additional power for URM and FG students. It is 
important to test whether UV intervention effects are moderated by 
prior performance, URM status, or FG status, in analytic models 
that include all three potential moderators (i.e., prior performance, 
URM status, and FG status), to determine whether the UV inter- 
vention has unique potential for URM and/or FG students. 

Indeed, there is reason to hypothesize that the UV intervention 
might promote performance for underrepresented students in 
STEM courses, over and above the positive effects documented for 
low-performing students in previous research. Goal congruity 
(Diekman, Brown, Johnston, & Clark, 2010), cultural mismatch 
(Stephens et al., 2012) and identity-based motivation (Oyserman & 
Destin, 2010) theories all suggest that congruity between a per- 
son’s identity, culture, or goals and the educational context can 
serve as a powerful motivational resource, and conversely, that a 
mismatch between personal goals and the educational context can 
lead to disengagement. Mismatch problems may be most acute in 
college STEM courses because the culture of science can be 
incongruent with the goals of at-risk students (Diekman, Clark, 
Johnston, Brown, & Steinberg, 2011). 

Smith, Thoman, and colleagues have found that communal 
goals such as working with, forming social connections with, or 


helping others are more frequently endorsed by URM college 
students, and that these types of goals are often perceived as 
inconsistent with the culture of science (Brown, Smith, Thoman, 
Allen, & Muragishi, 2015; Smith, Cech, Metz, Huntoon, & Moyer, 
2014; Thoman, Brown, Mason, Harmsen, & Smith, 2015). Re- 
search also suggests that FG students are more likely to have 
interdependent or “other-focused” goals in college (Harackiewicz, 
Canning et al., 2014; Piff, Kraus, Cote, Cheng, & Keltner, 2010; 
Stephens et al., 2012). Indeed, many researchers have noted that 
communion — a trait which reflects a greater emphasis on working 
with or helping others — is higher among ethnic minorities (Markus 
& Conner, 2013), and that Latino, Native American, and African 
American cultures especially emphasize helping members of their 
own communities (Fryberg & Markus, 2007; Harper, 2005; Smith 
et al., 2014; Torres, 2009). This analysis suggests that URM and 
FG students may struggle to stay engaged in STEM courses that do 
not seem to share their communal or “other- focused” goals. 

We hypothesize that a UV intervention may help underrepre- 
sented students find connections to important communal goals in 
STEM courses even if the courses do not emphasize communal 
themes in their curriculum (Brown et al., 2015). In other words, the 
UV intervention may provide underrepresented students the op- 
portunity to make course content congruent with their own goals. 
By giving students the opportunity to connect science content to 
their own lives in their own terms, they may be better able to 
identify the relevance of course topics to their personal goals, and 
become more motivated to engage with science content. Deriving 
meaningful connections between science and communal goals may 
be particularly important for students endorsing such goals, and 
thus the UV intervention may be uniquely powerful for underrep- 
resented students. 

Disentangling Race and Social Class in 
Intervention Research 

Both URM and FG students face disadvantages in college; 
however, there may be some challenges specific to URM students, 
and not FG students (e.g., racial discrimination), and some that are 
unique to FG students (e.g., fewer educational/ financial re- 
sources). Moreover, because race and social class are increasingly 
correlated in American society (Duncan & Murnane, 2011; Rear- 
don, 2011), some students (those who are both URM and FG) may 
experience both sets of challenges. Thus, students may face chal- 
lenges related to their URM status, their FG status, or both, and 
disadvantages may be compounded for FG-URM students (Jack, 
2014). A number of social-psychological intervention studies have 
addressed either racial gaps (e.g., Cohen et al., 2006; Sherman et 
al., 2013; Walton & Cohen, 201 1 ) or social class gaps in academic 
achievement (e.g., Harackiewicz, Canning et al., 2014; Smeding, 
Dumas, Loose, & Regner, 2013; Stephens et al., 2014). Given that 
social class and ethnicity are often confounded, it is not always 
clear whether interventions are addressing racial or social-class 
achievement gaps, or both. For example, Stephens Hamedani, and 
Destin (2014) tested a difference-education intervention (in which 
incoming students learned about how different educational back- 
grounds can impact college experiences) that improved perfor- 
mance for FG students, yet 38% of the FG students were also 
African American or Latino. Although the researchers controlled 


This document is copyrighted by the American Psychological Association or one of its allied publishers. 
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 


4 


HARACKIEWICZ, CANNING, TIBBETTS, PRINISKI, AND HYDE 


for ethnicity in their analyses, they did not consider the interactions 
between race and social class. 

Similarly, many interventions to close racial achievement gaps 
have not considered the impact of social class. In the Cohen et al. 
(2006) study testing a VA intervention with African American 
students, students’ social class (or parents’ educational attainment) 
was not reported nor was it included in analyses. In another VA 
study, Sherman et al. (2013) noted that virtually all of the Latino 
students were receiving lunch assistance (whereas few of the 
White students were), and that the racial gap was thus largely 
redundant with the social-class achievement gap. This overlap of 
race and social class in some populations makes it especially 
difficult to identify the groups for whom interventions are most 
effective. 

It should be possible, however, to examine the independent and 
interactive effects of race and social class with a diverse sample 
and adequate statistical power. One goal of the current research is 
to disentangle intervention effects associated with race from those 
associated with social class in a large-scale intervention study with 
college students in an introductory biology course. To explore 
these possibilities, we collected baseline measures of prior aca- 
demic performance, high school poverty rate, performance expec- 
tancies for the biology class, science background, psychological 
experiences, and motivation. Consideration of these variables may 
help us identify the psychological processes most relevant for 
interventions with particular groups of students (Walton, 2014), 
and help interpret differences in responsivity to social- 
psychological interventions among students who may have inter- 
secting identities. 

Intersectionality 

Intersectionality is a theoretical approach that simultaneously 
considers multiple categories of identity, difference, and disadvan- 
tage, such as gender, race, social class, sexual orientation, disabil- 
ity, and religion (Cole, 2009). This approach, originally based in 
critical race theory and feminist theory (Few-Demo, 2014), is 
critical of researchers’ tendency to consider a social category, such 
as African Americans, to be homogeneous, when in fact members 
of that category vary substantially on other dimensions such as 
social class and gender. In the research reported here, we consider 
the intersection of race and social class. 

Ferree (2010) noted that an approach is labeled intersectional if 
it considers multiple dimensions of inequality “and considers how 
they interactively define the identities and experiences ... of 
individuals and groups” (p. 428). There is some debate among 
intersectionality researchers regarding whether, when intersection- 
ality is combined with quantitative methods, it involves testing for 
multiple main effects of the intersecting dimensions, or whether it 
should involve statistical interactions between intersecting dimen- 
sions (Ferree, 2010). In the case of the intersection of race and 
social class, the main effects approach can detect the additive 
effects of race and social class, thereby disentangling these effects, 
which are so often confounded. The statistical interaction ap- 
proach, which is consistent with Ferree’s definition above, exam- 
ines whether certain combinations of race and social class (such as 
URM and FG status) have especially potent effects. In the research 
reported here, we test for both additive and interaction effects. 
Moreover, because intersectionality researchers have rarely used 


experimental methods, we present a novel test of intersectionality 
by testing a three-way interaction between the intervention and 
two intersecting person factors, race, and social class. 

The Present Study 

We tested two interventions, one that has proven to be effective for 
reducing some achievement gaps in middle-school and college classes 
(the VA intervention), and one that has proven to be effective for 
students with a history of low academic performance (the UV 
intervention). We address two primary questions about these in- 
terventions: (a) Can the UV intervention reduce achievement 
gaps?; and (b) Can the VA intervention be paired with a UV 
intervention? In other words, can a domain-general, identity-based 
intervention (VA) and a domain-specific, curriculum-based inter- 
vention (UV) be combined in a college biology course, and how 
might they work together? Such a combination might be additive 
or synergistic, with each boosting the effectiveness of the other, 
but these two interventions have not previously been combined, 
and it is unclear whether two writing-based interventions can be 
implemented effectively in a single course. 

In this study, we focused on closing achievement gaps in an 
introductory biology class, and tested UV and VA interventions in 
a 2 X 2 crossed design. Our design and a large sample allowed us 
to disentangle the independent and interactive effects of race 
(URM or Majority) and generational status (FG or CG) in mod- 
erating the effectiveness of the interventions. Although we focus 
mainly on social-psychological variables, we also included a 
social-structural variable, percent free/reduced lunch at the stu- 
dents’ high schools, as a proxy for poverty at both the school and 
neighborhood level. 

Method 

We implemented the UV and VA interventions in an introduc- 
tory biology course in a double-blind, randomized experiment at a 
large Midwestern university, across four semesters, in the first 
course of a two-course sequence. This foundational course, offered 
in both Fall and Spring semesters, is a prerequisite for 34 under- 
graduate biomedical majors at this university (e.g., biochemistry, 
neuroscience, nursing, zoology) and a critical gateway course for 
premedical preparation and further study in the biological sciences. 
Over these four semesters, 2,378 students were enrolled in this 
course, of whom 8% were underrepresented minority students 
(URM) and 21% were first-generation students (FG; i.e., no par- 
ent/guardian obtained a 4-year college degree). 1 All consenting 
URM and FG students enrolled in the course were included in this 
study, as well as a randomly selected subset of continuing- 
generation (CG) majority students (CG-majority students in this 
sample were 82% White, 18% Asian or Asian American). The 
remaining CG-majority students in the course received comparable 
assignments but were not included in the study. Of the 1060 
undergraduate students who were eligible for this study across four 
semesters, 1,040 (417 male, 623 female) completed the course and 
gave consent for access to their academic records (6 students did 
not consent, and 14 dropped the course). Participants were 423 


1 This distribution of FG and URM students was comparable to overall 
university demographics for the time period of this study. 
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CG-majority, 427 FG-majority, 126 CG-URM (51 African Amer- 
ican, 61 Hispanic, 14 Native American), and 64 FG-URM (26 
African American, 35 Hispanic, 3 Native American) students. 

The 15-week biology course covered three units: cellular biol- 
ogy, genetics, and either evolutionary biology (in Semesters 1 and 
2), or animal physiology (in Semesters 3 and 4). Students met three 
times per week for 50-min lectures. Between one and three lecture 
sections were offered each semester (for a total of eight lecture 
sections across the four semesters). In addition to lectures, students 
attended a 3-hr laboratory, led by a graduate teaching assistant, 
once each week. There were approximately 15-40 laboratory 
sections in each semester. Students also attended a 50-min recita- 
tion each week, led by a different graduate teaching assistant. 

Interventions 

Students were blocked on URM and FG status, gender, and 
lecture section and then randomly assigned to one of four condi- 
tions in a fully crossed 2X2 (UV Intervention X VA Interven- 
tion) experimental design. Instructors and teaching assistants were 
blind to experimental condition. 

UV intervention. Students completed either three UV or three 
control assignments. These writing assignments were fully inte- 
grated into the course and were presented as a course assignment 
from the instructors. Three weeks prior to each unit exam, course 
instructors e-mailed the assignment to each student. Students were 
given 5 days to complete each essay and turned them in via an 
online course management site. In both conditions, students were 
asked to: 

Select a concept or issue that was covered in lecture and formulate a 
question. Select the relevant information from class notes and the 
textbook, and write a 1-2 page essay. 

The utility-value (UV) assignment varied slightly across the four 
semesters, but all UV assignments asked students to answer their 
question using course material and discuss the relevance of the con- 
cept or issue to their own life or to the lives of others: 

Write an essay addressing this question and discuss the relevance of the 
concept or issue to your own life. Be sure to include some concrete 
information that was covered in this unit, explaining why this specific 
information is relevant to your life or useful for you. Be sure to 
explain how the information applies to you personally and give examples. 

In contrast, the control assignment instructed students to address 
their question by summarizing course material: 

Select the relevant information from class notes and the textbook, and 
write a one to two page response to your question. You should attempt 
to organize the material in a meaningful way, rather than simply 
listing the main facts or research findings. Remember to summarize 
the material in your own words. 

Biology graduate students were hired to grade each assignment on 
scientific merit and to ensure that students followed directions. Al- 
though some graduate student graders were also teaching assistants, 
graders were never assigned to grade essays of students who were in 
their laboratory or recitation sections. The fact that assignments and 
graded feedback were turned in via a course management site (and not 
during lecture or lab time) ensured that all teaching assistants re- 
mained blind to their students’ conditions. Grader feedback and essay 


grades were provided to each student a few days before the unit exam. 
Each assignment was worth 0.6% of the final grade in the course; 
1,034 students completed the first assignment, 1,017 students com- 
pleted the second essay assignment, and 1,006 students completed the 
third essay assignment (95% of students completed all three assign- 
ments). 

VA intervention. The VA intervention was administered in 
laboratory sessions early in the semester, and students wrote about 
personal values, as in previous research (Harackiewicz, Canning et al., 
2014; Miyake et al., 2010). Students in the VA condition were 
instructed to write about why two or three values, selected from a list, 
were important to them. Students in the control condition were in- 
structed to choose the two or three values that were least important to 
them, and to write about why other people might hold those values. 
Full methodological details regarding the implementation of V A are 
reported by Harackiewicz, Canning et al. (2014). 

Baseline Measures 

In the second week of the course, a questionnaire was administered 
in laboratory sections, with questions about attitudes about biology 
and demographic information. All questionnaire items were answered 
on a 7-point scale ranging from not at all true to very true or not at 
all to a lot, unless otherwise noted. Scale scores represent the mean of 
constituent items. Missing data (less than 1% on each measure) were 
handled by multiple imputation (Rubin, 1987). 

Attitudes about biology. Biology background was measured 
with three items (for each of three topics covered in the course: “I 
have a strong background in [cellular biology, genetics, and evolution 
or animal physiology];” a = .84). Belonging uncertainty (Walton & 
Cohen, 2011) was measured with two items (“When something bad 
happens, I feel that maybe I don’t belong at University X;” “Some- 
times I feel that I belong at University X, and sometimes I feel that I 
don’t belong at University X;” a = .83). Competence valuation 
(Harackiewicz & Sansone, 1991) was measured with two items (“It is 
important to me to do well in this course;” “I want to do well in 
Introductory Biology;” a = .71). Desire to contribute to society was 
measured with one item (“I want to study biology because I want to 
make a contribution to society”). Confidence about performance was 
measured with three items (“I am confident that I will do well in 
Introductory Biology;” “I expect to get a good grade in this course;” 
“I am confident that I can obtain a final grade of B or better in this 
course;” a = .82). Interest in biology was measured with five items 
(“I'm really looking forward to learning more about biology;” “Biol- 
ogy fascinates me;” “I think the field of biology is very interesting;” 
“I'm excited about biology;” “To be honest, I just don’t find biology 
interesting,” reversed; a = .93). Perceived utility value was measured 
with four items (“The material we are studying in this course is useful 
for everyone to know;” “This class is important to my future;” “I think 
what we are learning in Introductory Biology 151 is important;” “The 
study of biology is personally important to me;” a = .74). 

Motives for attending college. We administered a shortened 
version of Stephens, Fryberg, Markus, Johnson, and Covamibias’ 
(2012) scale in which students were asked to indicate which of 10 
items characterized their reasons for completing their college degree 
(checking as many as were relevant). Half the items referred to 
independent motives (i.e., “Become an independent thinker;” “Learn 
more about my interests;” “Prepare for a future career;” “Expand my 
understanding of the world;” and “Expand my knowledge of the 
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world”), whereas the other half referred to interdependent motives 
(i.e., “Help my family out after I’m done with college;” “Give back to 
my community;” “Provide a better life for my own children;” “Show 
that people with my background can do well;” and “Be a role model 
for people in my community”). Independent and interdependent mo- 
tives were measured by counting how many of the motives in each 
category were selected. In addition, we constructed a measure of 
helping motives by counting how many of three interdependent mo- 
tives that demonstrate the instrumentality of a college education for 
helping others (“Give back to my community;” “Help my family out 
after I’m done with college;” “Provide a better life for my own 
children”) were selected. 

Prior grade-point average (GPA) and high school poverty 
rate. We obtained students’ GPA from prior semesters from uni- 
versity records (n = 978). We also obtained information about the 
high school each student attended, in terms of the percentage of 
students who received financial assistance for school meals (percent 
free or reduced lunch) at those schools, as an indicator of poverty at 
the school or neighborhood level (n = 979). However, given that 
some students were freshmen or transfers, and that high school 
information was not available for all students, we were missing some 
data on both measures. Thus, we used multiple imputation (Rubin, 
1987) to create a measure of prior GPA and free/ reduced lunch (FRL) 
for all students. 

Outcome Measures 

Coding of articulated utility value and essay length. The 

utility-value and control writing assignments were coded for the level 
of utility- value articulated in each essay. Research assistants coded the 
assignments on a 0-4 scale based on how specific and personal the 
utility-value connection was to the individual. A “0” on this scale 
indicates no utility; a “1” indicates general utility applied to humans 


generically; a “2” indicates utility that is general enough to apply to 
anyone, but is applied to the individual; a “3” indicates utility that is 
specific to the individual; and a “4” indicates a strong, specific 
connection to the individual that includes a deeper appreciation or 
future application of the material. Utility-value scores from the three 
essays were summed to create an overall measure of articulated utility 
value. Interrater reliability with this coding rubric was high, with two 
independent coders providing the same score on 91% of essays. 
Disagreements were resolved by discussion. Research assistants also 
recorded the number of words in each essay so that we could test 
whether students wrote more in UV conditions, or if certain groups of 
students wrote more, possibly reflecting higher levels of task engage- 
ment. 

Biology course grade. Course instructors provided final 
course grades at the end of the semester (4.0 scale: A = 4.0, AB = 
3.5, B = 3.0, BC = 2.5, C = 2.0, D = 1.0, F = 0). Grading 
standards were consistent across sections and semesters. 

Results 

Overview 

Table 1 presents descriptive statistics and intercorrelations for all 
measures. Although students were randomly assigned to condition at 
the student level, we used hierarchical linear modeling to account for 
the nested structure of the data (students nested within eight lecture 
sections, across four semesters; Raudenbush & Bryk, 2002). We 
tested a two-level random-intercept model in which students were 
nested within the eight lecture sections, taking into account the inter- 
dependencies in the data — between students in the same lecture 
section — by estimating within and between component variance. The 
intraclass correlation coefficient was small; lecture sections accounted 
for only 1.67% of the variance in biology course grade. Although this 
analysis demonstrated that the nesting of students would not have a 
large effect on our analyses compared to multiple regression models, 


Table 1 


Zero-Order Correlations and Descriptive Statistics 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

1. Percent free/reduced 















lunch 

— 














2. Prior GPA 

3. Confidence about 

-.18” 

— 













performance 

-.04 

.14” 

— 












4. Interest in biology 

5. Perceived utility 

.00 

-.03 

.33” 

— 











value 

-.02 

-.03 

.26** 

.73” 

— 










6. Biology background 

-.05 

-.09” 

.28” 

.22** 

.19” 

— 









7. Belonging uncertainty 

.10” 

-.16" 

-.22** 

-.07* 

-.05 

-.10” 

— 








8. Competence valuation 

9. Desire to contribute 

-.01 

.06* 

.21” 

.31” 

.37” 

.03 

-.03 

— 







to society 

.01 

.03 

.26** 

.54** 

.56** 

.16” 

-.08* 

29 *. 

— 






10. Helping motives 

.13” 

-.09” 

.06 

.05 

.11” 

-.03 

.00 

.17” 

.15” 

— 





1 1 . Independent motives 

12. Articulated utility 

.01 

.02 

.08” 

.16” 

.12” 

-.05 

-.09** 

-.01 

.08** 

.15” 

— 




value 

-.06* 

.06 

-.01 

-.02 

.01 

-.06* 

-.03 

-.01 

.03 

.01 

-.02 

— 



13. Average essay length 

-.03 

20*. 

.06* 

.11” 

.08” 

-.02 

.00 

.07* 

.05 

.02 

.02 

.15” 

— 


14. Biology course grade 

-.21” 

.58” 

.23” 

.08* 

.05 

.08” 

_ 12*. 

.06 

.10” 

-.13” 

-.01 

.07* 

.27” 

— 

M 

26.66 

3.14 

5.80 

5.80 

5.78 

4.16 

3.22 

6.82 

5.60 

2.12 

3.62 

5.57 

485.93 

2.78 

SD 

16.94 

.52 

.87 

1.05 

.90 

1.32 

1.79 

.40 

1.49 

.92 

1.55 

4.07 

102.61 

.81 

N 

1040 

1040 

1040 

1040 

1040 

1040 

1040 

1040 

1040 

1040 

1040 

1040 

1040 

1040 


><.05. *><.01. 
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we modeled the nesting structure so that accurate standard errors 
would be obtained. We present comparisons of regression and HLM 
results for the primary analyses in Tables 2 and 3. Analyses with 
HLM and regression yielded consistent results. Regression results are 
reported here so that we can report effect sizes (betas). 


Background Differences for URM and FG Students 

We tested the main effects of URM status (majority = — 1, 
URM = 1), FG status (CG = — 1, FG = 1), and their interaction 
for each baseline measure, controlling for gender (female = 1, 


Table 2 


Effects of URM status, FG status, and Gender on Baseline Measures 



Hierarchical linear modeling 


Regression 



y 

F(df) 

P 

3 

t ( df) 

P 

Prior GPA 

URM 

-.16 

15.62 (1, 1035) 

.000 

-.13 

3.96(1035) 

.000 

FG 

-.16 

15.86(1, 1035) 

.000 

-.16 

3.99(1035) 

.000 

URM X FG 

.01 

.06(1, 1035) 

.813 

.01 

.23 (1035) 

.816 

Gender 

.06 

3.60(1, 1034) 

.058 

.06 

1.91 (1035) 

.057 

% Free/reduced lunch 

URM 

.27 

45.39(1, 1035) 

.000 

.21 

6.75 (1035) 

.000 

FG 

.34 

72.85 (1, 1035) 

.000 

.34 

8.55 (1035) 

.000 

URM X FG 

.14 

11.41 (1, 1035) 

.002 

.14 

3.38 (1035) 

.001 

Gender 

.00 

.01 (1, 1035) 

.670 

.00 

.08 (1035) 

.935 

Confidence about performance 

URM 

-.03 

.44(1, 1035) 

.509 

-.02 

.66(1035) 

.511 

FG 

-.02 

.32(1, 1035) 

.572 

-.02 

.56(1035) 

.579 

URM X FG 

.04 

.08 (1, 1035) 

.773 

-.01 

.29 (1035) 

.771 

Gender 

.25 

65.10(1, 1034) 

.000 

-.24 

8.08 (1035) 

.000 

Interest in biology 

URM 

.06 

1.88 (1, 1035) 

.170 

.05 

1.37 (1035) 

.170 

FG 

.05 

1.35 (1, 1035) 

.246 

.05 

1.16(1035) 

.245 

URM X FG 

.02 

.19(1, 1035) 

.661 

.02 

.44 (1035) 

.661 

Gender 

.00 

.01 (1, 1035) 

.910 

.00 

.11 (1035) 

.910 

Perceived utility value 

URM 

.07 

2.56(1, 1035) 

.110 

.05 

1.60(1035) 

.109 

FG 

.02 

.23 (1, 1035) 

.632 

.02 

.48 (1035) 

.631 

URM X FG 

.00 

.00(1, 1035) 

.992 

.00 

.01 (1035) 

.992 

Gender 

-.08 

7.14(1, 1035) 

.008 

.08 

2.68 (1035) 

.008 

Biology background 

URM 

-.09 

4.45 (1, 1035) 

.035 

-.07 

2.11 (1035) 

.035 

FG 

-.02 

.20(1, 1035) 

.655 

-.02 

.45 (1035) 

.654 

URM X FG 

-.02 

.26(1, 1035) 

.613 

-.02 

.51 (1035) 

.613 

Gender 

-.04 

1.89(1, 1035) 

.169 

.04 

1.38 (1035) 

.169 

Belonging uncertainty 

URM 

-.01 

.12(1, 1035) 

.732 

-.01 

.34(1035) 

.732 

FG 

.08 

3.86(1, 1035) 

.050 

.08 

1.97 (1035) 

.049 

URM X FG 

.01 

.08 (1, 1035) 

.775 

.01 

.29 (1035) 

.774 

Gender 

-.08 

5.86(1, 1035) 

.016 

.08 

2.42 (1035) 

.016 

Competence valuation 

URM 

.08 

4.12(1, 1035) 

.043 

.07 

2.03 (1035) 

.042 

FG 

.01 

.09(1, 1035) 

.761 

.01 

.31 (1035) 

.761 

URM X FG 

-.02 

.13 (1, 1035) 

.714 

-.02 

.37 (1035) 

.714 

Gender 

-.13 

15.82(1, 1035) 

.000 

.12 

3.98 (1035) 

.000 

Desire to contribute 

URM 

.08 

3.47 (1, 1035) 

.063 

.06 

1.87 (1035) 

.062 

FG 

.01 

.06(1, 1035) 

.808 

.01 

.24 (1035) 

.807 

URM X FG 

-.03 

.57 (1, 1035) 

.493 

-.03 

.69 (1035) 

.492 

Gender 

-.05 

2.25 (1, 1035) 

.134 

.05 

1.50(1035) 

.134 

Helping motives 

URM 

.20 

22.71 (1, 1035) 

.000 

.15 

4.77 (1035) 

.000 

FG 

.21 

26.08 (1, 1035) 

.000 

.21 

5.12 (1035) 

.000 

URM X FG 

.09 

5.27 (1, 1035) 

.022 

.09 

2.30(1035) 

.022 

Gender 

.08 

7.50(1, 1035) 

.006 

.08 

2.74 (1035) 

.006 

Independent motives 

URM 

.05 

1.49(1, 1035) 

.223 

.04 

1.25 (1035) 

.210 

FG 

-.00 

.00(1, 1035) 

.971 

-.00 

.03 (1035) 

.978 

URM X FG 

.07 

2.82 (1, 1035) 

.093 

.07 

1.69(1035) 

.091 

Gender 

-.09 

9.00(1, 1034) 

.003 

-.09 

2.98 (1035) 

.003 


Note. URM = underrepresented minority (URM = +1, majority = —1); FG = first-generation (FG = +1, 
continuing-generation = —1); Gender (female = +1, male = —1); GPA = grade-point average. 
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Table 3 


Effects of the UV Intervention, URM Status, and FG Status on Biology Course Grade 



Hierarchical linear modeling 


Regression 


7 

F(df) 

P 

3 

t (df) 

P 

UV intervention 

.06 

4.74(1, 1027) 

.030 

.08 

2.26 (1028) 

.024 

URM 

-.14 

25.54(1, 1025) 

.000 

-.13 

5.05 (1028) 

.000 

FG 

-.02 

.82(1, 1025) 

.364 

-.03 

.85 (1028) 

.393 

URM X FG 

.00 

.01 (1, 1024) 

.911 

.00 

.06 (1028) 

.950 

UV X URM 

.04 

2.75 (1, 1023) 

.097 

.06 

1.69(1028) 

.092 

UV X FG 

.01 

.11 (1, 1026) 

.737 

.01 

.24 (1028) 

.808 

UV X URM X FG 

.07 

6.33 (1, 1023) 

.012 

.08 

2.45 (1028) 

.015 

Confidence about performance 

.12 

36.95 (1, 1025) 

.000 

.15 

6.16(1028) 

.000 

Prior GPA 

.44 

456.36(1, 1025) 

.000 

.55 

21.32(1028) 

.000 

UV X Confidence 

.02 

1.46(1, 1024) 

.227 

.03 

1.27(1028) 

.203 

UV X Prior GPA 

-.04 

4.12(1, 1026) 

.043 

-.05 

2.01 (1028) 

.045 


Note. UV = utility- value intervention condition (UV = +1, control = —1); URM = underrepresented 
minority (URM = +1, majority = —1); FG = first-generation (FG = +1, continuing-generation = — 1); 
GPA = grade-point average. 


male = —1) on all baseline measures. 2 Table 2 presents the full 
regression and HLM results for all background variables. 

Prior GPA and high school poverty rate. On prior GPA, 
there were independent negative effects for both URM and FG 
status, (3 = — 0.13, p < .001, and (3 = — 0.16, p < .001, respec- 
tively, indicating both a racial achievement gap (Cohen’s d = 
0.27) and a social-class achievement gap (d = 0.31), which when 
considered together, indicate that FG-URM students had the low- 
est prior university GPAs (Figure 1, Panel A). On high school 
poverty rate (FRL), independent positive effects for both URM and 
FG status, (3 = 0.21, p < .001, and |3 = 0.34, p < .001, 
respectively, revealed that URM students attended more impover- 
ished schools than majority students, and that FG students attended 
more impoverished schools than their CG peers. Furthermore, the 
URM by FG interaction was also significant, p = 0. 14, p = .001. 
FG-URM students attended the most impoverished high schools 
and, by implication, lived in the most impoverished neighborhoods 
(see Figure 1, Panel B). 

Attitudes about biology and motives for attending college. 

There were no independent or interactive effects of URM or FG 
status on confidence about performance, interest in biology, per- 
ceived utility value, or independent motives for attending college, 
p > TOO. There was a negative effect of URM status on biology 
background, (3 = —0.07, p = .035, a positive effect of FG status 
on belonging uncertainty, p = 0.08, p = .049, as well as positive 
effects of URM status on competence valuation, p = 0.07, p — 
.042, and desire to contribute, p = 0.06, p = .062. On helping 
motives, we found independent positive effects for both URM and 
FG status, (3 = 0.15,p < .00 and p = 0.21, p < .001, respectively, 
as well as a URM by FG status interaction, p = 0.09, p = .022, 
indicating that FG-URM students had the strongest motivation to 
help their families and communities. 3 In fact, 80% of FG-URM 
students selected all three helping motives as reasons for attending 
college. Figure 2 presents these variables, standardized around the 
overall sample mean, for the four groups: CG-majority, FG- 
majority, CG-URM, and FG-URM. Considered together, these 
results show a unique pattern of challenges (higher high school 
poverty rate, lower prior GPA, weaker perceived biology back- 
ground, higher belonging uncertainty) and positive motivations 


(higher levels of competence valuation, desire to make a contri- 
bution, and helping motives) for the FG-URM students in this 
class. 

Gender differences. We found significant effects of gender 
on five of 11 baseline measures. Females reported lower levels of 
confidence about performance, p = — 0.24, p < .001, and higher 
levels of perceived utility value, p = 0.08, p = .008, competence 
valuation, p = 0.12, p < .001, belonging uncertainty, p = 0.08, 
p = .016, and helping motives as reasons for attending college, 
P = 0.08, p = .006, than male students. The effect of gender on 
prior GPA was not significant p = 0.06, p = .057, but women had 
slightly higher prior GPAs than men (d = 0.10). In contrast, men 
performed slightly better than women in this class {d = —0.16), 
but this difference was also nonsignificant, p = .22. 

Course Performance 

The primary outcome measure was course grade. Preliminary 
analyses revealed that there were no significant effects of the VA 
intervention for any group (i.e., there was no main effect of VA, no 
VA X URM Status interaction, no VA X FG Status interaction, 
and no VA X URM X FG interaction), and no significant inter- 
actions of VA with UV (these statistical tests are reported below). 
Therefore, we collapsed across VA condition for the analyses 
reported here, resulting in a two-cell UV versus control design. We 
used confidence about performance as a covariate to control for 
baseline performance expectations, and prior GPA as a covariate to 
control for prior performance. The basic model, which was tested 
for biology course grade and related variables, included 1 1 terms: 
the main effects of the UV intervention, URM status, and FG 
status, 3 two-way interactions (UV Intervention X URM Status, 
UV Intervention X FG Status, and URM Status X FG Status), 1 


2 We also tested all interactions with gender and group status and found 
no significant interactions on any baseline measure, except for high school 
poverty rate (% free/reduced-priced lunch). An interaction between gender, 
URM and FG status, (3 = —0.09, p = .025, revealed that FG-URM males 
came from the most impoverished high schools. 

3 This pattern also held for the five-item interdependent motives scale, of 
which helping motives were a three-item subset. 


CLOSING ACHIEVEMENT GAPS 


9 


o 

-O 


a. c 
T3 'a 
<u s 


Oh 

d 


-d 

H 


3.4 

3.3 

'ST 

75 3.2 
</> 

? 31 

a> 

<? 3 0 
o 

^ 2.9 

c 

o 2.8 

a. 

0) 

| 2.7 

<5 

2.6 

2.5 





CG-Majority FG-Majority CG-URM FG-URM 



CG-Majority FG-Majority CG-URM FG-URM 


Figure 1. Prior grade-point average (Panel A) and high school poverty rate (indicated by the percentage of 
students who received free or reduced-priced lunch; Panel B) as a function of URM (majority or URM) and 
generational status (CG or FG). Error bars are 95% confidence intervals. URM = underrepresented minority; 
CG = continuing generation; FG = first-generation. 


three-way interaction (UV Intervention X URM Status X FG 
Status), as well as the two covariates (confidence about perfor- 
mance and prior GPA), and 2 two-way interactions between the 
UV intervention and each of the covariates. 4 Including both co va- 
riates and their interactions with the UV intervention allowed us to 
test whether the effects found here are consistent with previous 
studies that found the UV intervention was particularly effective 
for students with low performance expectations (Hulleman & 
Harackiewicz, 2009) and students with low grades (Hulleman et 
al., 2010). Table 3 presents the results for course performance in 
regression and HLM. 

For course grade, there was a negative effect for URM status, 
(3 = -0.13, p < .001, showing that majority students obtained 
higher grades in the class than URM students. There were also 
positive effects of confidence about performance, p = 0.15, p < 
.001, and prior GPA, p = 0.55, p < .001, indicating that confident 
students and students with higher prior GPAs performed better in 
the course than students with lower confidence and students with 
lower prior GPAs, respectively. 



■ Biology Background ■ Belonging Uncertainty 

■ Competence Valuation ■ Desire to Contribute to Society 

■ Helping Motives 

Figure 2. Standardized baseline measures as a function of URM (major- 
ity or URM) and generational (CG or FG) status. URM = underrepresented 
minority; CG = continuing generation; FG = first-generation. 


A significant UV main effect, p = 0.08, p = .024, showed that 
the UV intervention improved performance slightly for all stu- 
dents, on average (M = 2.81, 95% Cl [2.74, 2.87]), relative to 
control (M = 2.76, 95% Cl [2.68, 2.83]), d = 0.06. In addition, 
course performance was somewhat higher for URM students in the 
UV condition relative to control, compared with majority students, 
P = 0.06, p = .092 for the two-way interaction of UV with URM 
status (Figure 3, Panel A). The performance gap for URM students 
was substantial in the control condition, d = .60, p < .001. The 
treatment effect for URM students was .20 grade points (d = 0.23), 
resulting in a 40% reduction in the racial achievement gap. 

However, this two-way interaction effect was qualified by a 
significant three-way interaction between the UV intervention, 
URM status, and FG status, p = 0.08, p = .015. The UV inter- 
vention was most effective for FG-URM students (Figure 3, Panel 
B). The performance gap for FG-URM students, relative to CG- 
majority students, was large in the control condition, .84 grade 
points, d = .98, p < .001. The treatment effect for FG-URM 
students was .51 grade points ( d = 0.55), resulting in a 61% 
reduction in the achievement gap for these students. 

UV replication analyses. The inclusion of the interactions 
between the UV intervention and both confidence about perfor- 
mance and prior GPA in our basic model allowed us to test for 
replication of prior utility-value research, and we found evidence 
for partial replication. Hulleman and Harackiewicz (2009) found 
that a UV intervention was most effective for students with low 
performance expectations. This finding was not replicated in our 


4 We tested all higher order interactions of the UV intervention, confi- 
dence about performance, prior GPA, URM status, FG status, and gender 
on course grade in a fully crossed model through the three-way level. There 
was not a significant effect of gender, nor were there any significant 
interactions between the intervention and gender; thus gender was ex- 
cluded from the basic model. The UV X URM X FG interaction remained 
significant when all higher order effects were included, and the effect size 
did not change (Yzerbyt et al., 2004). We found no significant three-way 
interactions apart from the one reported in the text, and we therefore 
trimmed the model accordingly (Cohen, Cohen, West, & Aiken, 2003). 
However, the 2 two-way interactions between the UV intervention and 
confidence and prior GPA were retained in the basic model to test for 
replication of prior findings. 
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Majority URM CG-Majority FG-Majority CG-URM FG-URM 

□ Control □Utility Value □ Control □Utility Value 

Figure 3. Course performance as a function of treatment condition (UV or control) and (A) URM status 
(majority or URM) and as a function of (B) both URM and generational status (CG or FG). Error bars are 95% 
confidence intervals. UV = utility- value; URM = underrepresented minority; CG = continuing generation; 
FG = first-generation. 


sample, p = 0.03, p = .203. However, there was a significant 
negative interaction between the UV intervention and prior GPA, 
P = —0.05, p = .045, indicating that the UV intervention was 
most effective for students who had the lowest prior GPAs. This 
finding replicates previous research (Hulleman et ah, 2010), and 
suggests that the UV intervention can help students with a history 
of poor performance, in addition to helping FG-URM students. 

Testing the effects of high school poverty rate. Thus far we 
have focused on one important aspect of social class, first- 
generation student status. However, social class encompasses both 
generational status and poverty levels. In order to test high school 
poverty as a predictor of academic performance and determine 
whether UV effects were moderated by high school poverty, we 
added several new terms to our basic model. Specifically, we 
added the main effect of high school percent free/reduced lunch 
(FRL), 2 two-way interactions with FRL (UV Intervention X FRL 
and URM Status X FRL), and the three-way interaction between 
the UV intervention, FRL, and URM status. In this model, there 
was a significant main effect of FRL on course performance, 
1(1024) = 3.30, p = .001, p = —.10, showing that students from 
poorer high schools obtained lower grades in the course. However, 
there were no significant interactions of FRL with either URM 
status or the UV intervention, ps > .30. In contrast, the positive 
main effect of the UV intervention remained significant, f(1024) = 
1.93, p = .05, P = .07, as well as the interaction between the UV 
intervention and prior GPA, 1(1024) = 2.00, p = .05, P = —.05. 
Most important, the UV X FG X URM interaction remained 
significant, 1(1024) = 2.19, p = .03, p = .08, with FRL con- 
trolled. 5 These analyses suggest that although high school poverty 
and generational status both influenced academic performance, the 
UV intervention was effective for FG-URM students even when 
FRL was taken into account. 

Mediation Analyses 

To understand the motivational processes underlying the effects 
of the UV intervention, we first examined the content of students’ 
essays in terms of articulated utility value, with the basic model 
described above. As expected, students in the UV condition artic- 


ulated more utility value (i.e., made more personal connections to 
curricular content) in their essays (M = 8.76, 95% Cl [8.57, 8.95]) 
than controls (M = 1.56, 95% Cl [1.44, 1.69]), P = 0.87, p < .001. 
This important manipulation check indicates that our curricular 
intervention was successful in encouraging students to make per- 
sonal connections with the course material in their writing assign- 
ments. There was also a significant effect of prior GPA, p = .05, 
p = .001, and a significant UV X GPA Interaction, p = 0.06, p < 
.001; students with higher prior GPAs articulated more utility 
value in their essays, particularly in the UV condition. However, 
there were no significant interactions with FG or URM status (p > 
.40), suggesting that the effect of the intervention was comparable 
for all groups, in terms of helping students articulate the utility 
value of course topics. 

Next, we examined the length of students’ essays with the same 
regression model. There was a main effect of prior GPA, p = 0. 19, 
p < .001; students with higher prior GPAs wrote longer essays. In 
addition, students wrote longer essays in the UV condition (M = 
506 words, 95% Cl [476.99, 534.49]) compared with control (M = 
479, 95% Cl [469.23, 488.39]), p = 0.11, p = .008, suggesting 
that the UV intervention promoted engagement with curricular 
content, averaged across all students. A three-way interaction 
between the UV intervention, URM and FG status, p = 0.12, p = 
.004, revealed that FG-URM students showed the largest interven- 
tion effect, writing longer essays in the UV condition (M = 506 
words, 95% Cl [476.99, 234.49]) than in control (M = 433, 95% 
Cl [389.10, 476.32]), suggesting that they became particularly 
engaged with this assignment. Compared with their peers in the 
control condition, FG-URM students in the UV condition wrote, 
on average, about 73 more words in their essays (see Figure 4). 


5 We also tested higher order interaction terms, but none were signifi- 
cant. The effects of FRL were not limited to course performance. Ancillary 
analyses revealed a significant effect of FRL on prior GPA, f(1034) = 4.23, 
p < .001, p = —0.13, controlling for URM and FG status, which continued 
to be significant predictors, f(1034) = 3.03, p = .002, (3 = —0.10 and 
f(1034) = 2.80, p = .005, [3 = —0.12, respectively. This suggests that the 
effects of high school poverty on college grades are independent of the 
effects of URM and FG status. 



This document is copyrighted by the American Psychological Association or one of its allied publishers. 
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 


CLOSING ACHIEVEMENT GAPS 


11 



□ Control DUV 

Figure 4. Average essay length as a function of treatment condition (UV 
or control), URM status (majority or URM), and generational status (CG or 
FG). Error bars are 95% confidence intervals. UV = utility-value; URM = 
underrepresented minority; CG = continuing generation; FG = first- 
generation. 

Considered together, these analyses suggest that students in the 
utility-value condition made more personal connections to the 
course material in their essays (as requested by the utility-value 
assignment) than in control conditions, but that FG-URM students 
became particularly involved in the process of writing about utility 
value, writing longer essays in the intervention condition. A crit- 
ical question is whether essay length (an indicator of cognitive 
engagement) mediated the effects of the UV intervention on per- 
formance for FG-URM students, who wrote more words and who 
benefited most from the intervention, in terms of course perfor- 
mance. 

Moderated mediation analysis. We tested the indirect effects 
of the UV intervention on course grade through essay length as a 
function of URM and FG status, in a test of moderated mediation 
(Hayes, 2013, Model 12). Table 4 shows the effects of UV inter- 
vention, URM status, FG status, and essay length on course grade 
and the indirect effect for each student group (i.e., CG-majority, 
FG-majority, CG-URM and FG-URM). The confidence interval 
testing the indirect effect of the UV intervention through essay 
length does not include zero for FG-URM (95% Cl [0.010, 0.076]) 
and CG-Majority (95% Cl [0.004, 0.028]) students. 6 Thus we can 
conclude that essay length partially mediated the positive effect of 
the UV intervention on course grade for these two groups of 
students, with the largest indirect effect for FG-URM students, 
who wrote longer essays and benefitted most from the UV inter- 
vention. 

Exploratory Text Analyses 

To investigate essay content more fully, we used Linguistic 
Inquiry and Word Count (LIWC) text analysis to determine 
whether the content of the essays varied as function of the UV 
intervention and URM or FG status. LIWC software calculates the 
degree to which people use different categories of words (e.g., 


personal pronouns, social processes, cognitive mechanisms) in 
their writing by calculating how many words from each LIWC 
dictionary appear in each document. (Pennebaker, Booth, & Fran- 
cis, 2007). 

Utility-value assignments are designed to encourage students to 
write differently, in terms of both style and content, than when 
they are simply summarizing the course material (as in the control 
writing assignments). For example, UV assignments are designed 
to promote personal writing, which is expected to contain more 
references to the self and/or other people (i.e., social language) and 
use more “everyday” language than typical scientific writing. In 
addition, the process of making connections between course ma- 
terial and their own lives is expected to deepen students’ cognitive 
processing and increase engagement with the material, which 
should also be reflected in the content of students’ essays. 

We selected nine LIWC dictionaries that might capture the style 
and content of UV writing. We hypothesized that the UV assign- 
ment would prompt the use of more personal pronouns and shorter 
words (i.e., simpler words and less technical vocabulary), as in- 
dexed by the LIWC personal pronouns (e.g., I, us, your) and longer 
words (> 6 letters) dictionaries. 

We used the social processes dictionary (e.g., advice, discuss, 
encourage), and its three subcategories — family words (e.g., mom, 
daughter, brother), friend words (e.g., friend, buddy, neighbor), 
and human words (e.g., adult, baby, boy) — to explore the social 
content of students’ essays. These dictionaries assess relationships 
and communication with others (Tausczik & Pennebaker, 2010). 
Finally, to explore content related to cognitive involvement, we 
chose the cognitive mechanism dictionary (e.g., cause, conclude, 
explain) and two subcategories — insight words (e.g., consider, 
idea, understand) and causal words (e.g., because, effect, hence). 
The insight dictionary assesses active learning, encoding, and 
understanding, and the causal dictionary connotes attempts to 
explain causes and effects (Klein & Boals, 2001; Pennebaker & 
King, 1999; Pennebaker & Stone, 2003). 

We examined the number of words used from each of the nine 
LIWC categories as a function of the UV intervention, URM 
status, and FG status. In addition to the 11 terms of our basic 
model, our model for these exploratory text analyses also con- 
trolled for gender and the UV X Gender interaction, because 
previous research using LIWC has found gender differences in 
word usage (e.g., Newman, Groom, Handelman, & Pennebaker, 
2008). For these analyses we were interested in the main effect of 
the UV intervention, to determine if students wrote differently in 
UV essays, compared with control essays. We were also interested 
in the UV X URM X FG interaction, to explore whether any 
treatments effects differed for FG-URM students. The regression 
results are shown in Table 5. 

As expected, students in the UV condition used more personal 
pronouns, (3 = 0.82, p < .001, and fewer long words (> 6 letters), 
(3 = — 0.16, p < .001. In addition, UV essays contained more 
social words, p = 0.53, p < .001, family words, p = 0.37, p < 
.001, friend words, p = 0.11, p = .009, and human words, p = 
0.25, p < .001, than control essays. There were also more cogni- 
tive mechanism words, p = .21, p < .001, and insight words, p = 


6 We also tested for moderated mediation of the UV X GPA effect on 
performance by essay length, but found no evidence for mediation. 
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Table 4 

Essay Length as a Mediator of the UV Intervention Effect 


Predictor 

B 

SE 

t 

P 

Essay length 





UV intervention 

.11 

.04 

2.66 

.008 

URM status 

-.06 

.04 

1.36 

.174 

FG status 

.02 

.04 

Al 

.642 

URM X FG 

.00 

.04 

.03 

.980 

UV X URM 

.06 

.04 

1.37 

.171 

UV X FG 

.05 

.04 

1.08 

.279 

UV X URM X FG 

.12 

.04 

2.91 

.004 

Confidence about performance 

.04 

.03 

1.24 

.214 

Prior GPA 

.19 

.03 

6.08 

.000 

UV X Confidence 

.01 

.03 

.20 

.839 

UV X Prior GPA 

.01 

.03 

.23 

.819 

Course grade 





Essay length 

.11 

.02 

5.45 

.000 

UV intervention 

.05 

.03 

1.84 

.067 

URM status 

-.13 

.03 

4.89 

.000 

FG status 

-.03 

.03 

.94 

.345 

URM X FG 

.00 

.03 

.06 

.952 

UV X URM 

.04 

.03 

1.48 

.140 

UV X FG 

.00 

.03 

.06 

.952 

UV X URM X FG 

.05 

.03 

1.98 

.048 

Confidence about performance 

.12 

.02 

6.03 

.000 

Prior GPA 

.42 

.02 

20.22 

.000 

UV X Confidence 

.03 

.02 

1.26 

.208 

UV X Prior GPA 

-.04 

.02 

2.08 

.038 

Mediator Student status 

Boot indirect effect 

Boot SE 

Boot LLCI 

Boot ULCI 

Essay length CG-majority 

.01 

.01 

.004 

.028 

Essay length FG-majority 

.00 

.01 

-.014 

.008 

Essay length CG-URM 

.00 

.01 

-.016 

.018 

Essay length FG-URM 

.04 

.02 

.010 

.076 


Note. UV = utility-value (UV = +1, control = — 1); URM = underrepresented minority status (URM = +1, 
majority = —1); FG = first-generation status (FG = +1, continuing-generation = —1); GPA = grade-point 
average. 


.33, p < .001, in UV essays, suggesting that students’ writing in 
the UV condition was more characteristic of active cognitive 
engagement in the course content (Pennebaker & Francis, 1996; 
Pennebaker & King, 1999), but there were no significant effects on 
causal words. Considered together, these results indicate that the 
utility-value essays contained more personal writing and social 
themes, as well as greater evidence of cognitive engagement than 
control essays. 

Interestingly, significant UV X URM X FG interactions on 
social words, |3 = .11,/? = .002, family words, (3 = .08,/? = .043, 
cognitive mechanism words, (3 = .10, p = .02, and insight words, 
(3 = .11, p = .005 (see Figures 5 and 6) indicated that FG-URM 
students, who wrote longer UV essays and benefitted most from 
the UV intervention, wrote essays that contained more social 
words, as well as more words indicative of cognitive involvement 
with the course material. Consistent with our finding that FG- 
URM students are especially oriented toward helping their families 
and communities, their UV essays contained more family words 
(but not more friend or human words, suggesting a relative em- 
phasis on family). 7 

Moderation Analyses 

Another way to gain insight into the dynamics of the UV 
intervention is to examine moderation by individual-difference 


variables. Our primary model revealed that the UV intervention 
was effective for students with low prior performance, as has been 
shown in previous research (Hulleman et al., 2010). Given that the 
intervention had a positive effect for FG-URM students controlling 
for this interaction with prior GPA, our results suggest that the 
efficacy of the UV intervention for FG-URM students is not 
completely explained by the efficacy of the UV intervention for 
students with low prior GPA. 

To explore whether UV effects might be moderated by the 
background variable of helping motives, we tested whether the UV 
intervention was effective, more generally, for students who en- 
dorsed helping motives as a reason for attending college, regard- 
less of prior GPA, ethnicity or generational status. Given that there 
was so little variance in helping motives among FG-URM students 
(80% of them selected all three helping motives as reasons for 


7 We tested whether these differences in essay content for FG-URM 
students mediated the effects of the intervention on performance with the 
same moderated-mediation model reported in Table 4, by testing all five 
potential mediators (i.e., word count, social words, family words, cognitive 
mechanism words, and insight words) first individually and then simulta- 
neously. Word count was the only significant mediator, indicating that 
increased engagement in the material (as indexed by longer essays) rather 
than the specific content of the essays, accounted for the UV intervention 
effects on course performance for FG-URM students. 
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Table 5 


Exploratory Text Analysis 



Personal pronouns 

Six-letter words 

Words 

per sentence 

Social words 


Family words 


3 

t 

P 

3 

t 

P 

3 

t 

P 

3 

t 

P 

3 

t 

P 

UV intervention 

.82 

34.12 

.000 

-.16 

4.02 

.000 

.11 

2.58 

.010 

.53 

14.40 

.000 

.37 

9.43 

.000 

URM 

.02 

.95 

.342 

-.02 

.64 

.520 

-.03 

.92 

.361 

.05 

1.56 

.118 

.02 

.77 

.443 

FG 

.00 

.13 

.900 

.02 

.36 

.718 

-.08 

1.93 

.053 

.08 

2.09 

.037 

.05 

1.27 

.205 

URM X FG 

.00 

.13 

.895 

.02 

.49 

.627 

-.06 

1.46 

.145 

.05 

1.41 

.158 

-.01 

.19 

.852 

UV X URM 

-.01 

.23 

.815 

.07 

1.80 

.073 

.09 

2.02 

.043 

.04 

1.02 

.307 

.01 

.23 

.817 

UV X FG 

.02 

.96 

.340 

.02 

.37 

.712 

-.02 

.38 

.703 

.07 

1.94 

.053 

.07 

1.67 

.095 

UV X URM X FG 

.02 

.95 

.343 

.07 

1.69 

.092 

.00 

.04 

.972 

.11 

3.07 

.002 

.08 

2.02 

.043 

Confidence 

.02 

.82 

.413 

.01 

.33 

.740 

-.02 

.45 

.653 

-.02 

.73 

.464 

-.02 

.63 

.530 

Prior GPA 

-.03 

1.66 

.098 

.25 

8.08 

.000 

-.01 

.15 

.879 

.03 

.89 

.375 

.06 

1.84 

.066 

UV X Confidence 

-.01 

.51 

.610 

-.03 

.83 

.410 

.01 

.32 

.747 

.01 

.50 

.619 

-.03 

1.00 

.317 

UV X Prior GPA 

.04 

2.25 

.025 

-.04 

1.29 

.198 

.04 

1.21 

.225 

.01 

.41 

.682 

.02 

.72 

.472 

Gender 

-.01 

.49 

.624 

.03 

.93 

.354 

-.10 

2.95 

.003 

-.04 

1.38 

.169 

.09 

2.84 

.005 


Friend words 


Human words 

Cognitive mechanism 
words 

Insight words 


Causal words 



3 

t 

P 

3 

t 

P 

3 

t 

P 

3 

t 

P 

3 

t 

P 

UV intervention 

.11 

2.61 

.009 

.25 

5.98 

.000 

.21 

5.03 

.000 

.33 

8.18 

.000 

.01 

.22 

.825 

URM 

.04 

1.13 

.258 

.02 

.66 

.512 

-.03 

.93 

.352 

.04 

1.36 

.174 

.02 

.51 

.611 

FG 

-.09 

2.15 

.032 

-.03 

.65 

.514 

.05 

1.11 

.267 

.03 

.60 

.546 

.04 

.96 

.337 

URM X FG 

-.04 

.88 

.380 

-.02 

.56 

.576 

.05 

1.13 

.259 

.03 

.76 

.450 

.03 

.69 

.493 

UV X URM 

-.02 

-.35 

.727 

.00 

.10 

.923 

.02 

.35 

.724 

.03 

.68 

.495 

.02 

.40 

.688 

UV X FG 

.03 

.68 

.497 

.02 

.59 

.558 

.05 

1.18 

.237 

.07 

1.62 

.105 

.00 

.06 

.953 

UV X URM x FG 

.02 

.41 

.680 

.02 

.52 

.607 

.10 

2.33 

.020 

.11 

2.83 

.005 

.05 

1.10 

.271 

Confidence 

.04 

1.12 

.265 

-.04 

1.15 

.251 

.04 

1.35 

.178 

.03 

.83 

.408 

.05 

1.52 

.129 

Prior GPA 

.00 

.05 

.960 

.00 

.01 

.996 

.11 

3.31 

.001 

.03 

.84 

.400 

.13 

3.84 

.000 

UV X Confidence 

.00 

.09 

.925 

-.06 

1.89 

.060 

.02 

.60 

.552 

-.04 

1.17 

.242 

-.05 

1.54 

.125 

UV X Prior GPA 

.04 

1.30 

.193 

.02 

.55 

.583 

.01 

.25 

.802 

.04 

1.21 

.226 

-.04 

1.14 

.255 

Gender 

.07 

2.27 

.024 

.04 

1.24 

.216 

-.05 

1.59 

.112 

-.03 

.96 

.339 

-.04 

1.14 

.256 


Note, df = 1027. UV = utility-value intervention condition (UV = +1, control = —1); URM = underrepresented minority (URM = +1, 
majority = —1); FG = first-generation (FG = +1, continuing-generation = —1); Confidence = confidence about performance; gender (female = +1, 
male = —1); GPA = grade-point average. 


attending college), however, we could not test models that in- 
cluded these students. We therefore excluded FG-URM students in 
this analysis in order to test whether the UV intervention improved 
performance for other students who endorsed helping motives. 
Accordingly, we tested a variation of the basic model: in this 
exploratory model, we did not include the terms testing the FG X 
URM and UV X FG X URM interactions (because there were no 
FG-URM students in this analysis) and we controlled for the effect 
of gender and the UV X gender interaction (because gender was a 
significant predictor of helping motives in the baseline analyses). 
This resulted in a model with 13 terms: the main effects of the UV 
intervention, helping motives, URM status, and FG status, and 3 
two-way interactions (UV Intervention X URM Status, UV Inter- 
vention X FG Status, and UV Intervention X Helping Motives), as 
well as three covariates (confidence about performance, prior 
GPA, and gender), and their two-way interactions with the UV 
intervention. 8 

Of interest in this model was whether helping motives predicted 
performance or moderated the effects of the UV intervention. We 
found a negative effect of helping motives, (3 = —0.09, p = .001, 
indicating that students who selected more helping motives as 
reasons for attending college received lower grades in the course. 
In this model, the main effect of the UV intervention was not 
significant, (3 = -0.01, p = .813 (but recall that FG-URM 


students are omitted from the analysis), but we found a positive 
interaction between the UV intervention and helping motives, |3 = 
0.05, p = .047, indicating that the UV intervention was most 
effective for students who endorsed more helping motives. Nota- 
bly, the UV X Prior GPA interaction was somewhat reduced, but 
still nearly significant, (3 = —0.04, p = .094, indicating that the 
helping moderation observed in this model was independent of the 
prior GPA moderation documented earlier. This result supports our 
hypothesis that the UV intervention can be particularly effective 
for students who are motivated by a desire to help others. 

Failure to Replicate Previous Values 
Affirmation Finding 

In a previous study in the same context, we had tested the VA 
intervention alone (i.e., without implementing the UV interven- 
tion) in a single semester, and found that it improved performance 
for FG students, relative to CG students (Harackiewicz, Canning et 
al., 2014). This effect did not differ according to URM status, and 
the VA intervention did not affect performance for URM students. 
In the present study, in which the UV and VA interventions were 


8 We also tested higher order interaction terms, but none were signifi- 
cant. 
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Figure 5. Essay content reflecting social processes, as indicated by social words (Panel A) and family words 
(Panel B) as a function of treatment condition (UV or control), URM status (majority or URM), and generational 
status (CG or FG). Error bars are 95% confidence intervals. UV = utility-value; URM = underrepresented 
minority; CG = continuing generation; FG = first-generation. 


both implemented across four semesters, the positive effect of VA 
for FG students was not replicated. Table 6 presents the model 
testing the UV intervention crossed with the VA intervention and 
shows that there were no significant effects of VA, and that all the 
UV effects reported previously remained significant when VA 
terms were included in the model. Here we consider four possible 
reasons for this nonreplication: (a) the earlier finding could have 
been a false positive; (b) continued administration of the VA 
exercise over many semesters may have reduced its effectiveness; 
(c) the size of achievement gaps may have been different in the 
earlier study than in the current one; and (d) students had more 
writing assignments in the current study than in the earlier study, 
and the UV intervention, which involved more writing, could have 
outweighed the effects of the VA intervention. 

Although it is possible that the earlier finding was a false 
positive, the results were consistent with previous work in college 
courses (Miyake et al., 2010), as well as a conceptual replication 
of several earlier studies in different contexts (Cohen et al., 2006; 
Cohen & Sherman, 2014). Indeed, the VA intervention has proven 
effective at addressing achievement gaps in various educational 


contexts. However, there have been some previous failures to 
replicate these effects (e.g.. Dee, 2014; Kost-Smith et al., 2012), 
and it is important to explore other factors that might account for 
nonreplication. 

Although the VA intervention was administered with methods 
identical to those employed in this same context by Harackiewicz, 
Canning et al. (2014), it is possible that the continued administra- 
tion of VA (and UV) interventions over a 3-year period in the same 
course led to some communication between students or teaching 
assistants that weakened the impact of the interventions in this 
study. This issue may be more critical in the case of “stealth” 
interventions such as the VA intervention, however, because com- 
munication about the intervention can undermine treatment effi- 
cacy (Cohen et al., 2012; Cohen & Sherman, 2014). Curricular 
interventions such as the UV intervention may be more robust to 
such effects. We monitored the administration of both interven- 
tions carefully, and detected no problems, but in a large course 
with multiple sections of large lectures, many instructional staff, 
and several discussion and laboratory sections, there may have 



CG-Majority FG-Majority CG-URM FG-URM 
□ Control ®UV 


CG-Majority FG-Majority CG-URM FG-URM 
□ Control 0UV 


Figure 6. Essay content reflecting cognitive involvement in course material, characterized by cognitive words 
(Panel A) and insight words (Panel B), as a function of treatment condition (UV or control), URM status 
(majority or URM), and generational status (CG or FG). Error bars are 95% confidence intervals. UV = 
utility- value; URM = underrepresented minority; CG = continuing generation; FG = first-generation. 
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Table 6 

Effects of the VA Intervention, the UV Intervention, URM Status, and FG Status on Biology 
Course Grade 


HLM Regression 

7 F(df) p 0 t(df) p 


UV intervention 

URM 

FG 

URM X FG 

UV X URM 

UV X FG 

UV X URM X FG 

Confidence about performance 

Prior GPA 

UV X Confidence 

UV X Prior GPA 

VA intervention 

VA X URM 

VA X FG 

VA X URM X FG 

UV X VA 

UV X VA X URM 

UV X VA X FG 

UV X VA X URM X FG 


.06 

4.52(1, 1017) 

.034 

-.14 

24.81 (1, 1016) 

.000 

-.03 

.89(1, 1015) 

.346 

.00 

.01 (1, 1015) 

.943 

.04 

2.56(1, 1013) 

.110 

.01 

.13 (1, 1016) 

.715 

.07 

6.52(1, 1013) 

.011 

.12 

36.86(1, 1015) 

.000 

.44 

452.68(1, 1015) 

.000 

.02 

1.17(1, 1014) 

.280 

-.04 

4.03 (1, 1017) 

.045 

-.01 

.07(1, 1013) 

.788 

-.02 

.36(1, 1015) 

.551 

-.05 

2.93 (1, 1013) 

.087 

-.04 

2.37(1, 1015) 

.124 

-.00 

.00(1, 1014) 

.949 

.02 

.44(1, 1014) 

.506 

.04 

2.52(1, 1014) 

.113 

.01 

.06 (1, 1016) 

.808 

.01 

.31 (1, 1014) 

.577 

-.00 

.00(1, 1016) 

.950 


.07 

2.21 (1018) 

.027 

-.13 

4.98(1018) 

.000 

-.03 

.89(1018) 

.375 

.00 

.02(1018) 

.983 

.06 

1.63 (1018) 

.103 

.01 

.28(1018) 

.778 

.08 

2.49(1018) 

.013 

.16 

6.16(1018) 

.000 

.55 

21.25 (1018) 

.000 

.03 

1.15 (1018) 

.252 

-.05 

1.99(1018) 

.047 

-.01 

.25 (1018) 

.801 

-.02 

.61 (1018) 

.539 

-.06 

1.89(1018) 

.072 

-.05 

1.58(1018) 

.115 

-.00 

.04(1018) 

.969 

.02 

.68(1018) 

.498 

.06 

1.67(1018) 

.096 

.01 

.29(1018) 

.772 

.01 

.54(1018) 

.592 

-.00 

.12(1018) 

.907 


VA X Confidence 
VA X Prior GPA 


Note. UV = utility-value intervention condition (UV = +1, control = —1); URM = underrepresented 
minority (URM = +1, majority = —1); FG = first-generation (FG = +1, continuing-generation = —1); VA = 
values affirmation (VA = +1, control = —1); GPA = grade-point average. 


been undetected factors that detracted from implementation effi- 
cacy (Cohen et al., 2012). 

Another possibility is that achievement gaps differed between 
the samples. We examined achievement gaps in course grades in 
the control condition of each study, to examine performance in the 
biology class for students who received neither intervention (25% 
of students in the current study, n = 226) or did not receive a VA 
intervention (50% of students in the Harackiewicz, Canning et al. 
(2014) study, n = 396). We conducted 2X2 regression analyses 
of the effects of FG status, URM status, and their interaction, on 
course grade, controlling for gender and confidence about perfor- 
mance, in the control group of each sample. Table 7 shows course 
performance for all four groups of students in the control condi- 
tions of each study. In the Harackiewicz, Canning et al. (2014) 
sample, we found a significant effect for FG status, (3 = —0.30, 
p < .000, and a significant effect for URM status, 0 = -0.14,p = 
.006. In the current sample, we found that the FG effect was not 
significant, (3 = —0.07, p = .417, but that the URM effect was, 
(3 = —0.19, p = .005. These analyses indicate that the perfor- 
mance difference between majority and URM students in the 
control condition (i.e., the racial achievement gap in this course) 
was comparable in the two studies (mean grade point difference of 
.38 and .36, respectively), but that the difference in performance 
between FG and CG students (i.e., the social-class achievement 
gap in this course) in the control condition was much larger in the 
Harackiewicz, Canning et al. (2014) study than in the current study 
(mean grade point difference of .48 and .08, respectively). This 
occurred mainly because FG controls in the current study per- 
formed considerably better than those in the previous one; in 
addition, CG students in the current study performed slightly less 


well than those in the previous study. It is likely that VA is more 
effective when achievement gaps are larger, and previous research 
substantiates this view (Hanselman. Bruch, Gamoran, & Borman, 
2014). 

It is also possible that differences in the educational context 
account for differences in treatment efficacy. Although the struc- 
ture and content of the biology course was constant across the two 


Table 7 

Course Performance in the Control Group Compared With 
Harackiewicz, Canning et al. (2014) Sample 

Biology course grade (control group only) 

Harackiewicz, 
Canning et al. 

Current sample (2014) 



Mean ( SD ) 

N 

Mean (SD) 

N 

All CG students 

2.77 (.79) 

121 

2.86 (.69) 

319 

All FG students 

2.69 (.87) 

105 

2.38 (.85) 

77 

Mean social class gap (CG-FG) 

.08 


.48** 


All majority students 

2.80 (.82) 

183 

2.80 (.74) 

363 

All LTRM students 

2.44 (.81) 

43 

2.42 (.79) 

33 

Mean race gap (Majority-URM) 

.36** 


.38** 


CG-majority 

2.85 (.76) 

93 

2.87 (.69) 

298 

CG-URM 

2.50 (.85) 

28 

2.67 (.66) 

21 

FG-majority 

2.75 (.88) 

90 

2.45 (.84) 

65 

FG-URM 

2.33 (.75) 

15 

2.00 (.85) 

12 


Note. CG = continuing-generation; FG = first-generation; URM = 
underrepresented minority. 

’><. 01 . **><. 001 . 
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studies, there may have been differences in the way the course was 
taught by different instructors, and this may have influenced how 
the VA intervention worked. In fact, one important feature of the 
course differed by design: in the present sample, three writing 
assignments (the UV/control essays) were added to the curriculum. 
Thus students were required to do more writing in the biology 
course in the present study compared to the same course in the 
Harackiewicz, Canning et al. (2014) and this may have changed 
students’ experiences of the course. It is possible that the VA 
intervention was less effective because the course was more 
writing-intensive or seemed to place a greater emphasis on per- 
sonal writing. Given that the UV and the VA interventions both 
involve personal writing, it also seems possible that the extensive 
writing in the three UV essay assignments outweighed the brief 
VA writing exercises conducted in laboratory sessions in the 
present study, thereby dampening the effect of the VA interven- 
tion. This suggests that it may not work to combine different types 
of writing interventions in a single semester (Walton, Logel, 
Peach, Spencer, & Zanna, 2014). 

Our results, considered together with other nonreplications of 
VA effects (Dee, 2014; Kost-Smith et al., 2012), suggest that 
although values affirmation can have powerful positive effects, it 
is sensitive to contextual and sample differences in ways that we 
do not yet fully understand (Walton, 2014; Yeager & Walton, 
2011). More research is needed to identify factors that moderate 
the effectiveness of the VA intervention in college contexts, and 
future research should also examine how VA interventions work in 
conjunction with other interventions. 

Discussion 

There have been many attempts to address achievement gaps in 
high school and college classes, but researchers have typically 
focused on just the racial achievement gap or just the social class 
gap (Cohen et al., 2006; Harackiewicz, Canning et al., 2014; 
Sherman et al., 2013; Stephens et al., 2014; Walton & Cohen, 
2011). To our knowledge, this is the first intervention study 
designed to close achievement gaps that has also been designed to 
disentangle the effects of race from the effects of social class. Our 
findings illustrate the importance of analyzing the separate and 
interactive effects of URM and FG status when evaluating inter- 
ventions intended to close achievement gaps. Focusing on one gap 
to the exclusion of the other overlooks the challenges and moti- 
vational patterns unique to each group. We found that our UV 
intervention was particularly helpful for a unique group of under- 
represented students — FG-URM students — who came into this 
course with the lowest grades, highest levels of high school pov- 
erty, and weakest biology background. In other words, the UV 
intervention was most effective for the students who began the 
course with the most disadvantages. 

On average, the UV intervention promoted performance for all 
students. Thinking about how course topics relate to their own life, 
or the lives of others, made biology more relevant to students, and 
helped them stay engaged in their coursework. Students wrote 
longer essays when thinking about utility value, and this may 
reflect greater engagement with course topics (Elliot & Klobucar, 
2013). The assignment may also have promoted deeper cognitive 
processing of the material, facilitating learning (Harackiewicz, 
Tibbetts et al., 2014), and this is supported by the text analysis 


findings. Students in UV conditions used more cognitive mecha- 
nism words, especially insight words, which Pennebaker and King 
(1999) argue reflect active thinking and learning. Considered to- 
gether, these results contribute to a growing body of work showing 
that curricular interventions based on psychological principles can 
promote active engagement and performance in college science 
classes (Deslauriers, Schelew, & Wieman, 2011; Haak, HilleRis- 
Lambers, Pitre, & Freeman, 2011). 

For interventions to close achievement gaps, however, the ef- 
fects must be strongest for disadvantaged groups, and we found the 
largest UV intervention effect for FG-URM students in this study. 
This result highlights the potential of the UV intervention to 
address important achievement gaps in college classes. Moreover, 
content analyses revealed that writing about utility value was 
particularly effective for FG-URM students. Although all students 
articulated more utility value in UV conditions, the process of 
articulating utility value was particularly effective for FG-URM 
students. They wrote longer essays in the UV condition, and 
mediation analyses revealed that this increased engagement with 
course material contributed to their increased performance in the 
course. Moreover, LIWC text analyses revealed that their essays 
contained more words indicative of active thinking, which may 
help account for their improved performance in these conditions 
(Klein & Boals, 2001). 

First-Generation Underrepresented Minority Students 

What accounts for the greater responsivity of FG-URM students 
to the UV intervention? Our analyses provide some insight into the 
mechanisms at work for these students. One possibility is that the 
intervention was most effective for FG-URM students because UV 
interventions are most effective for students with lower prior 
grades (Hulleman et al., 2010), and FG-URM students had the 
lowest prior GPA among the categories of students. Notably, the 
positive effect of the UV intervention for FG-URM students re- 
mained significant after accounting for the effect of prior GPA and 
for the positive effects of the intervention for students with lower 
prior grades. This suggests that the UV intervention may work to 
promote engagement and close achievement gaps for multiple 
reasons that go beyond the issue of prior performance. 

For example, the UV intervention may also work by helping 
students connect course material to important personal goals. Our 
moderation analysis, in which we tested whether individual dif- 
ferences in helping motives moderated the effects of the UV 
intervention, revealed that the UV intervention was more effective 
in improving performance for students who endorsed helping 
motives. This suggests that the UV intervention may be particu- 
larly impactful for FG-URM students, who endorsed more helping 
motives for attending college than any other group of students. 

Consistent with previous research, FG-URM students in this 
study were highly motivated to succeed in the course, and they 
were particularly motivated by communal goals, such as helping 
others, contributing to society, or giving back to their families and 
communities (Smith et al., 2014; Smith, Brown, Thoman, & Die- 
ner, 2015; Stephens et al., 2012). From the perspective of both 
Stephens’ cultural mismatch theory and Diekman’s goal congruity 
theory, the UV intervention may have been especially powerful for 
FG-URM students because it gave them the opportunity to explore 
connections between the course content and their communal goals, 
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thereby alleviating a sense of goal incongruity or cultural mis- 
match. Indeed, the LIWC text analyses revealed that FG-URM 
students in the UV condition were most likely to include content 
related to social processes, and family words in particular, in their 
essays. Discovering and appreciating the utility value of curricular 
content for their communal values and goals may, in turn, have 
increased their engagement with curricular content (as evidenced 
by their longer essays and more frequent use of cognitive engage- 
ment words in UV conditions), and inspired them to work harder 
in the course. Rather than addressing a deficit, the UV intervention 
may work for these students by relating the curriculum to students’ 
strengths — their positive motivations for studying biology. 

In contrast, the UV intervention did not help FG-majority stu- 
dents, and our baseline analyses offer some insight into why our 
intervention was more effective for some groups than others. Our 
findings suggest that FG students (both majority and URM) tended 
to feel more uncertainty about belonging, and we would not expect 
a UV intervention, which is specific to curricular content, to 
impact a student’s more general sense of belonging in college. For 
these students, targeting their uncertainty about belonging might 
prove more effective (Ostrove & Long, 2007; Walton & Cohen, 
2011). Interventions that target social belonging processes, such as 
Walton and Cohen’s (2011) belonging intervention, focus on stu- 
dents’ experiences or sense of fit in the academic environment 
(Walton, 2014), and might be more effective with FG students. 
Indeed, Stephens et al. (2014) found that a difference-education 
intervention in which freshmen FG students heard stories about 
positive FG student experiences was effective in helping FG 
students adjust to college and perform better in their classes. 

Different interventions address different psychological pro- 
cesses, and it is important to consider the motivational patterns of 
the students targeted by interventions. Given that different inter- 
ventions may not work well together (Walton et al., 2014), it is 
especially important for intervention researchers to understand the 
characteristics of the group they hope to help, in order to identify 
the most effective interventions to implement with that particular 
group. Only by considering the independent and interactive effects 
of race and social class were we able to identify the unique 
background and motivational profile of FG-URM students and 
document the full potential of the UV intervention to promote 
performance for the underrepresented students most at risk in this 
course. 

The Intersection of Race and Social Class 

We tested both the additive and interactive effects of race and 
social class, as well as the three-way interaction of race, social 
class, and the UV intervention, on academic and motivational 
variables and found evidence for both the main effects and the 
interactional approach to intersectionality. We measured 11 base- 
line variables, including prior GPA, high school poverty rate, 
confidence, baseline interest, and helping motives, and found a 
mixed pattern of results. On some measures (e.g., baseline interest, 
confidence) we found no group differences; on others, we found a 
single main effect (e.g., URM students reported lower levels of 
biology background than majority students and FG students re- 
ported higher levels of belonging uncertainty than CG students). 
On other measures, we found two main effects but not an inter- 
action (e.g., prior GPA), supporting the additive model of inter- 


sectionality. On the measures of high school poverty and helping 
motives, however, we found two main effects and a significant 
interaction, supporting the interactive model. The interaction effect 
on high school poverty, for example, indicates that FG-URM 
students came from more impoverished schools and neighbor- 
hoods than either their race or generational status would predict. 
The interaction effect on helping motives suggests that FG-URM 
students are particularly motivated to help others and give back to 
their communities. Only by measuring a number of baseline vari- 
ables were we able to gain a nuanced understanding of differences 
between student groups and intersecting categories. These findings 
indicate the power of this statistical approach to identify important 
intersectionalities, and may contribute to development of more 
effective interventions in the future. 

With respect to the primary outcome of course grade, we were 
able to explore additive and interactive models of intersectionality 
in moderating treatment effects. On course performance, we found 
a main effect of race, but not FG status, nor a URM X FG 
interaction. Perhaps most interesting, the three-way interaction 
with the intervention was significant (UV X URM X FG, Table 4, 
Figure 3), demonstrating that intersectionality can apply to the 
effects of an intervention. That is, the intervention was differen- 
tially effective as a function of both the race and social class of the 
students and was particularly beneficial for FG-URM students. 

This same three-way interaction was also found for essay 
length; the intervention had a strong effect for FG-URM students, 
who became particularly engaged in the utility-value writing as- 
signments. In the variables derived from LIWC analyses, this 
three-way interaction was found for social-process words, family 
words, cognitive-mechanism words, and insight words, in all cases 
showing the pattern that the intervention was especially potent for 
FG-URM students. Overall, then, the findings point to the impor- 
tance of considering the intersection of social categories in inter- 
vention research. Whereas prior gap-closing intervention studies 
have focused on single social categories, our results suggest that 
this approach may neglect important within-group differences that 
only an intersectional approach can fully address. By considering 
students’ intersecting identities, intervention researchers can de- 
velop a more nuanced understanding of both the populations they 
seek to help and the effectiveness of their interventions. 

Implications for Expectancy- Value Theory 

In this study we attempted to identify variables that may account 
for differential effects of the UV intervention, and our findings 
suggest that the relationships are complex. We found that the UV 
intervention was moderated by social class, race, prior GPA, and 
helping motives. The UV intervention implemented in an intro- 
ductory biology course was effective for FG-URM students (a 
doubly disadvantaged category of students; Jack, 2014), for stu- 
dents with low prior GPAs (an academic background category), 
and for students with high helping motives (a motivational cate- 
gory). These findings extend the range of groups who can benefit 
from a UV intervention, and contribute to our understanding of 
how UV interventions work, and to expectancy-value theory more 
generally. 

By testing both prior performance and baseline performance 
expectations (confidence about performance) as moderators of the 
UV intervention effects in the same model, we were able to test 
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which variable was more important in this context. We found that 
prior performance was the more critical moderator of UV effects, 
whereas performance expectancies did not moderate the effects of 
the UV intervention. This helps us understand the expectancy side 
of the expectancy-value model (Eccles, 2009). Prior research has 
employed one moderator or the other; in laboratory studies with 
new learning activities, for example, researchers typically test 
self-reported expectancies as the moderator, because there is no 
history of prior performance to consider (Canning & Harackie- 
wicz, 2015; Shechter, Durik, Miyamoto, & Harackiewicz, 2011). 
In classroom studies, researchers have used either self-reported 
performance expectations (Hulleman & Harackiewicz, 2009) or 
prior exam performance (Hulleman et al., 2010) to assess expec- 
tancies. Expectancies are of course shaped by prior performance 
experiences (Eccles, 2009; Pajares, 1996), but one measure reflects 
performance history and the other is more forward-looking. In an 
introductory college course, however, students may not be able to 
gauge how well they will do in the course (in fact, we found no 
group differences in confidence at baseline, despite eventual group 
differences in course grades), and thus prior performance may be 
the more relevant moderator in this context. Therefore, to deter- 
mine which expectancy variables are important moderators of UV 
intervention effects, it is important to consider both self-reported 
expectancies and prior performance in any given context. 

Furthermore, our mediation and content analyses provide some 
insight into UV mechanisms, and here we address the value side of 
the expectancy-value model. Our results help us understand how 
writing about utility value may help to promote performance — by 
increasing engagement in the writing assignments, by providing 
opportunities to write differently (i.e., including more personal and 
social content), and by encouraging active thinking about course 
topics. Although were able to show that engagement in the writing 
assignment (measured in terms of essay length) mediated course 
performance, most of our other analyses were more exploratory, 
and highlight a need for future research. One important question 
concerns the connections between these mechanisms. For exam- 
ple, in writing UV essays, do students first find personal examples 
and then become more engaged with course content, or does 
greater engagement help students make more personal connec- 
tions? Are these separate processes, and how might we measure 
them more objectively? It is difficult to conduct fine-grained 
process analyses in a large-scale classroom intervention study, and 
future research may require controlled laboratory studies to ad- 
vance our understanding of the motivational mechanisms that 
underlie the effectiveness of the intervention. Overall, this research 
contributes to a better understanding of how students think about 
value in science classes, and illuminates the value side of 
expectancy-value theory, which is understudied compared with the 
expectancy side (Harackiewicz, Rozek, Hulleman & Hyde, 2012). 
Our results show the power of utility value in enhancing students’ 
academic achievement, and highlight new areas for future re- 
search. 

Strengths and Limitations 

Although our sample was large enough to disentangle the effects 
of racial and generational status, sample sizes of different racial/ 
ethnic groups were not large enough to explore differences be- 
tween racial groups. In particular, we considered African Ameri- 


can, Latino, and Native American participants together in a single 
URM category, and were not able to separate these groups in 
analyses because of limited sample sizes. Our findings point to the 
importance of considering intersectionality in social science re- 
search, and it will be particularly important for future research to 
address intersectionality on a more specific level (e.g., the inter- 
section of African American and FG identities). Different inter- 
ventions may be more or less effective for different racial and 
cultural groups, and these are important issues to pursue in future 
research. 

Similarly, the use of first-generation college student status as a 
measure of social class captures an important aspect of social class 
in the higher education context, but a dichotomous measure does 
not capture the full range of parental educational levels that may 
contribute to students’ experiences in college. Furthermore, FG 
status does not capture all aspects of social class that may con- 
tribute to achievement gaps. For example, limited financial re- 
sources and the need to work part- or even full-time while attend- 
ing college are major barriers for low socioeconomic status (SES) 
students (Pascarella, Pierson, Wolniak, & Terenzini, 2004) and we 
would not expect a psychological intervention to address these 
financial barriers. Indeed, by including high school free/reduced 
lunch (as a proxy for poverty at the school and neighborhood 
levels) as a predictor, we found that high school poverty had an 
independent negative effect on performance, over and above the 
effects of FG and URM status. Nonetheless, the effects of FG and 
URM status remained, as well as the positive effect of the UV 
intervention for FG-URM students. These issues will be important 
to address in future research with more direct measures of income 
and poverty level, as well as ways to capture high school quality 
directly. 

Finally, although we inferred that FG-URM students in our 
sample perceived an incongruity or mismatch between their mo- 
tivations and values and the norms of science, we did not collect 
measures of students' perceptions of incongruity or mismatch. We 
hypothesized that the UV intervention provided FG-URM students 
the opportunity to make course content congruent with their own 
values and goals, and indeed, the treatment effects on social 
process and family words for FG-URM students suggests that the 
content of their essays was congruent with communal goals. How- 
ever, we were not able to measure whether the UV intervention 
changed perceptions of incongruity, and thus our analysis remains 
speculative. If the UV intervention is particularly powerful be- 
cause it helps FG-URM students resolve a mismatch between their 
communal goals and their perceptions of scientific fields, it will be 
important to measure these perceptions as well as the psycholog- 
ical experience of mismatch in future research. 

Conclusions 

Racial and social class achievement gaps are a major issue in 
higher education. When these gaps occur in introductory science 
courses, they can lead URM and FG students to abandon STEM 
majors and careers. Our utility-value intervention successfully 
reduced the achievement gap for URM students by 40% and for 
FG-URM students by 61%. Moreover, the intervention was de- 
signed for implementation within the curriculum in large-lecture 
classes, making it practical for widespread implementation at low 
cost. This simple curricular change — asking students to reflect on 
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personal connections to course material in course assignments — 
could help students remain engaged in introductory courses and 
ultimately widen the STEM pipeline. 
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